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ABSTRACT 

Programmatic instructional developnent refers to 
sequenced and coordinated efforts to produce effective instructional 
programs which cumulate over tine and which attain outcomes that 
would be impossible under ncn-progrannatic projects. As practiced at 
the southwest Regional Laboratory (SWRL) , it involves the combined 
efforts of specialists from such fields as education, the social and 
behavioral sciences, and the information sciences. SHRL has 
consistently defined an instructional product as "organized methods 
and materials which accomplish specified instructional outcomes under 
natural conditions." An instructional product becomes an 
instructional program with the addition of the human resources 
support system and the instructional management system required to 
effect instructional improvement. The staff at SWRL recognizes that 
the program one is currently producing is always third best; the 
second best is the one you are working on, but is not yet 
deliverable, and the first best reflects ideas that are bright and 
promising for the future. Programmatic instructional development, 
coupled with the required continuous direct allocation of personnel 
and financial resources, results in a high degree of replicability 
and *n actual attainment of reliable increments in the instructional 
effectiveness of a given program. (JY) 
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ABSTRACT 

A discussion of the implications and potentials of programmatic 
instructional development as an area of professional endeavor. Indi- 
cations of the rapidly advancing sophistication and complexity of the 

field are presented. 
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P RO GRAMM AT I C INSTRUCTIONAL DEVELOPMENT 1 



Programmatic instructional development is a relatively new area of 
professional endeavor which is rapidly advancing in sophistication and 
complexity. The recent popularity and fluid growth of the endeavor 
create certain difficulties for this presentation. As is the custom 
education, several of the key concepts in the endeavor have been °Y^ 
generalized and distorted faster than they could be uSiroal* 

Terms such as product, system, formative evaluation ; a " d " 

management are being used with promiscuous abandon to try to j^nd n 
status to tired traditional practices. Thus, there is a strong ±±keLx 
hood that the meaning of some of the terms I’ll be using as a re 
been inflated out of all value. 

Mv article, "The Nature of Educational Development," (1970) was an 
aL tempt to counteract thin operation of Gresham's Law of educational 
fads That article was intended to define the general oun 
programmatic instructional development. The present remarks will ad- 
dress personal implications and potentials of such efforts. 

Point one. The field of instructional development requires more, 

rather than less, general intellectual resources and f SP ?^nf ^tric- 
ciniine competence from an individual than does the field of instruc 
tional research. Although some people at’, beginning to call themselve 
educational de^lopers, the term "develope -" is an unfortunate one, and 
persons who use it tend to be as shallow professionally as Person .who 
call themselves educationists. Programmatic instructional developme 
involves a division of labor among highly competent P ro ^si°na £ 
cialists, each doing his own thing in the interest of clCco_p g 

common specified outcome. 

Educational researchers have tended to be Independent °P er ^ors . 

The style has been medieval rather than modern. Individual researc^ 
barons over time surround themselves with few or many 

and some command the allegiance of a band of knights in gluing armor 
who do battle in the broader world. The results of such efforts ac 
cumulate, but they do not cumulate. This condition has been generally 
acknowledged by the educational research community within the past few. 
years, but efforts to do something about it are still in _ 7 

It is an open question how fast it will be possible to accomplish in 

human enhancement endeavors what it took a couple of ra yaa * s 

I^plish in the science and technology generated during the industrial 

revolution . 



!a version of this paper was presented at ^.Annuai Meeting of the 
American Educational Research Association, Minneapolis, March 5, 
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My point is that treating the lore and knowledge of educational 
rpuparch as directly relevant to educational development is a serious 
error \ °„e-^ developer will inevitably be superficially trained and 
llll operate superficially. Interdependent, coordinated, and sequenced 
efforts of highly competent professional specialists are require 1 
a modern development effort. These efforts cannot be effectively con 
ducted by prima donna generalists who 'all themselves developers. 

The level of personal methodological competence and substantive spe 
c -1 ' alization of *-he individuals involved is as important in determining 
the quality of a development as of a research effort. Now, and ikely 
forever, such high quality persons are likely to be trained and ° 
identify themselves as discipline specialists rather than as developers. 

Although educational researchers can shift to development efforts 
lust as physicists can shift to engineering efforts, a somewhat d e<. 
ent personal orientation is required. It is unfortunate that the 
hypothesis-testing perspective has so strongly influenced educational 
research methodology. It is, of course, quite feasible to cast either 
research or development activities in-o a hypothesis t ? St ^ n | 

To do so, however, leads to a formalism and methodological dogmatism 
which usually takes precedence over the outcomes of the activity. 

It is far more productive to view both research and development 
as uncertainty reducing rather than hypothesis testing processes. 
Uncertainty may be associated either with abstract concepts and their 
interrelationships or it may be associated with real world accomplis - 
ments. Procedures directed to reduce the uncertainty associated with 
relationships among abstract concepts — theories — may be termed re 
search. Procedures directed to reduce the uncertainty in accomplishing 
natural world outcomes may be termed development. Se Y er ^ 3 
between research and development endeavors are shown in Table 1 . While 
the table lists distinguishing differences, I want to emphasize that 
the researcher operating in a development context retains all of the 
intellectual challenges of the research context and adds a few. Not 
only does development effort generate reports, it also contributes 
directly to the creation of organized methods and materials which can 
accomplish socially useful ends . This is a personal satisfaction which 
is only gained by wishful extrapolation in a research context.. 

The distinctions • e tween researcher, "developer," and programmatic 
development have consequences for manpower training as well as for 
the conduct of development efforts. In my view, it would be a serious 
error to establish special university training programs designed to 
prepare "educational developers." The requirement for specialists with 
sound methodological training in various research areas and with 
experience in educational development activities will certainly increase 
However, the need is for specialists, not generalists. The specialist 
requirements are not limited to education ana the social and behavior 
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TABLE 1 
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science disciplines . Specialists are also required from the infor- 
mation sciences, linguistics, business, engineering, and the health 
fields. I would not rule out the hard sciences or the humanities, 
but at the doctoral level, the chief value of such specialists in 
an instructional development effort is to specify and sequence desired 
instructioiiai outcomes. Unfortunately, such people tend to view their 
capability as extending more broadly, and this severely reduces their 
net as well as their gross potential contribution. 

The conscious division of labor in programmatic instructional 
development has a couple of other consequences which are worth pointing 
ou t. First, it is possible to make very effective use of non-doctoral 
persons to accomplish many of the tasks conventionally handled by the 
doctoral level researcher. At SWRL we run a ratio of about two bach- 
elors— masters people and one secretarial— technical person for each 
doctoral level person. It is possible to have highly competent artists, 
printers, computer programmers, school liaison personnel, editors, 
story writers, test constructors, statisticians, audiovisual and in- 
strumentation specialists, experiment conductors, as well as general 
"graduate student type apprentice" assistants. This means that rather 
than a do-it-yourself generalist, it is possible for each doctoral 
person to function as a specialist, taking advantage of the skilled 
services of other specialists . 

Under these circumstances, one begins to view personal time 
priorities from a different perspective. The priority is no longer 
a hot idea. Out staff generates hot ideas at a dime a dozen. Each 
hot idea, or hypothesis if you like, could be the basis for a study. 

The criterion for deciding whether the potential study should be 
performed is not that it would generate a project grant, but that it 
would likely yield the moot useful information in terms of time and 
cost to reduce the un certainty associated with developing some aspect 
of an instructional program. Thus, one is able to think much bigger, 
move much faster, and over time see more subs cantiation of his efforts 
in programmatic instructional development than in most other areas of 
education at the present time. 

Perhaps it is now time to indicate more precisely what I mean by 
the term programmatic. The term "programmatic" as used here refers 
to sequenced and coordinated efforts which cumulate over time and 
which attain outcomes that would be impossible under non— programmatic 
projects. It is true that small project support, when added to an 
ongoing p ub licly supported institution such as a school district or 
university, can generate many separate activities. On a straight 
cost comparison of number of individual activities , "little science 
is inherently a better bargain than "big science." The problem is 
that little science is also inherently uncoordinated and noncumulative. 
Unfortunately , Educational R&D is still all little science. The 
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Educational Laboratories are viewed by some as "big science," but they 
have been financially starved from the beginning. Lee DuBridge has 
remarked thac the biggest lesson learned by the physical scientists 
during World War IX was how to spend money. Unfortunately, the phys 
ical scientists tended to overdo their learning and the backlash on 
this is currently affecting R&D support generally, including that for 
educational R&D. 

In many respects, educational R&D appears to be in much the same 
shape as the physical sciences in the pre-World War II period. Whether 
there is any further parallel in the history remains to be seen. It 
is ironic that the War on Poverty, which accompanied the 1965-68 growth 
of educational R&D, lacked a number of the sustaining scientific growth 
properties of World War II, which propelled the physical sciences to a 
prominent level. Rising social expectations have exceeded social ac- 
complishments. While the original expectations, in retrospect, were 
excessive, so now are the current disappointments excessive, but they 
are creating a revolution of rising frustrations. 

Daniel Lerner, the political scientist, has noted an analogous 
situation in a very different domain — the recent history of the 
developing countries of the world following Marshall Plan and Point 
XV efforts of the United States in the 19b0 u ! 



"The spread of frustration in areas developing less 
rapidly than their people wish can be seen as the 
outcome of a deep imbalance between achievement and 
aspiration. In simple terms, this situation arises 
when manv people in a society want far more than 
they can hope to get. ... A serious imbalance in 
this want— get ratio characterizes areas beset by 
rising frustrations. Typically, in these situations, 
the denominator increases faster than the numerator; 
that is , aspiration outruns achievement to such a 
degree that many people, even if they are making 
some progress toward their goal, are dissatisfied 
because they get much less than they want. Indeed, 
in some developing countries aspirations have risen 
high enough to annul significant achievements in the 
society as a whole" (1969, pp • 189—90). 



It remains to be seen whether the current Federal priority to stem 
the revolution of rising frustrations will result in a regressive 
reaction for education R&D. However, it should lead f to -long— range 
advancement as responsible persons begin to recognize that the solution 
of social problems must give high priority to the knowledge problem; 
of supporting sustained efforts to generate knowledge that permits 
problem solutions. President Nixon's announcement of; the National 
Institute of Education is an extremely hopeful sign in this regard,. 
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Meanwhile, let's get back to the ranch. One hears a good deal 
of talk these days about educational products. SWKL has contributed 
to this talk, but we do not hold ourselves responsible for the way 
the term "product" is being used by others. SWKL has consistently 
defined an instructional product as "organized methods and materials 
which accomplish specified instructional outcomes under natural con- 
ditions Thus, a product represents the organized wherewithal for 

reliably accomplishing socially desirable ends. We have been careful 
to distinguish products from people. We have also been careful to 
distinguish products from unorganized materials and practices which 
do not reliably produce specified results. This eliminates consider 
able confusion. It is also important to recognize that instructional 
products do not an instructional program make . Two additional re 
quirements must be considered: the human resources support system and 

the instructional management system required to effect instructional 
improvement . 

In addition to students and teachers, SWRL— developed ins true— 
tional progr ams make specific provision for parents, tutors, and aides; 
principals, supervisory and curriculum specialists, pupil personnel 
specialists, and district administrators; development agency and 
monitoring support agency personnel. 

This structure is also the basis for the instructional management 
system and the installation training system. The instruction is 
computer managed. En route criterion-referenced tests are automatically 
scored and analyzed and reports are generated for the teacher indica_ing 
student performance and suggesting supplementary instruction as appro- 
priate . Cumulative summary reports are also generated for the various 
other groups included in the human resources network . 

The currently available supply of knowledge and qualified people 
in instructional product development is small. Most of the recent 
literature concerning the field is rhetorical, not operational, and if 
you take a close look at the actual work of people who are espousing 
fancy models of development, ycu find little relationship between their 
model and their activity. It is easy to talk in general of a design- 
test-re test-produce model, but the what-and-how-to of this very general 
paradigm does not follow in a straightforward deduction. It must be 
learned the hard way . 

While physical analogies such as automobiles and bridges are 
better than nothing, they wear very thin, very quickly. The chief 
limitations with these product analogies, so far as I'm concerned, is 
not that they are inherently faulty, but that they are simplistic -- 
they don't go far enough. It is true that we need educational products 
analogous to automobiles and bridges. A few of these have actually^ 
been generated in the past couple of years. That is, I can identify 
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organized methods and materials for you which will dependably accomplish 
specified instructional consequences under natural real-world what we 
call hands-off — conditions. 

However, the automobile functions in a complex environment of 
support — highways, the petroleum industry, etc. The automobile also 
has a complex delivery network — marketing, advertising, etc. To 
consider the automobile out of this context is simplistic. Similarly, 
it is simplistic to consider instructional products without considering 
support system and delivery system requirements. 

It is easy to become overwhelmed by the apparent complexity of 
such support and delivery system requirements in an educational context. 

To expect instant change in current educational marketing and training 
institutions is obviously as silly as to expect it of public schools. 

The temptation is to branch either to despair and withdrawal or to 
dismay and revolution. Examples of persons pursuing each branch are 
now dangerously prevalent throughout society. Neither route can be 
expected to effect a solution. The trick appears to be to recognize 
the complexity and allocate reasonable personnel resources to reduce 
the complexity over time. The seven reconmendations recently set forth 
by Patrick Moynihan (1970) appear eminently reasonable. 

"First, put first-rate minds to work." What constitutes a first- 
rate mind" is relative, but if in doubt, figure you should keep looking. 

"Second, establish measurements of problem situations that get as 
close as possible to what it is you are trying to achieve." Secondary 
outcomes are fine, but they are not substitutes for primary results. 
Direct measures of significant educational outcomes are not that hard 
to come by if you don't get intellectually tied up in "the criterion 
problem" or in psychometric trivia. 

"Third, begin an experimental mode." This point is not necessary 
to embellish for this group. Moynihan’s reaction is to non-replicable 
anecdotes or specially selected situations such as gifted teachers • 

"Fourth, be on guard for social aggression masking as social commit- 
ment. One of the better ways of doing competitors in is to assert a 

superior concern for either the general welfare, or else the welfare of 
some specifically deprived group. Behind this facade there go^sIWli the 
bloodiest form of ethnic competition, individual aggrandizement and 
group aggression in general." 

"Fifth, do not expect instant results. ...Hard as it may be for 
the ardent heart to believe this , overpromising hurts everyone. 

"Sixth, think of processes, not institutions" of what goes on 
in and outside of the schools, not of the schools per se. 
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"Finally, be of good cheer." Dramatic improvement is unlikely. 

Small, steady moves in a clearly comprehended direction may be possible. 

While these points could be regarded as "happy talk," I consider 
them useful attitudinal guidelines for programmatic instructional 
development . 

I do not wish to create the impression that instructional develop- 
ment technology is presently vacuous. Far from it. This is a rapidly 
advancing area of knowledge. Consider Table 2. The contents of the 
table are accurate but are easily misinterpreted. First, the stages 
are not linear. That is, one does not first start and complete formu 
lation, then start and complete prototypes, and so on. Concern for 
and attention to the installation and program stage are present from 
the beginning. If none of the uncertainty associated with the various 
stages were present, it would be possible to go to the program stage 
immediately. While each stage feeds the others in a dynamic sequential 
fashion, the sequence is not a linear vector. 

None of the stages ever really ends. For example, before the SWRL 
staff have completed any one formulation, the results are already obso- 
lete as far as they are concerned. This dynamism can easily create an 
insurmountable personal and management dilemma in a continuous operation. 
Some form of task completion is vital for both personal and institutiona 
reasons. The resolution is to recognize that the program one is cur- 
rently producing is always third best. The second best is the one you 
are working on, but is not yet deliverable. The first best reflects 
the work you would rather be doing: the rough ideas which are always 

bright and promising in an untested form and which, over time and test 
ing phase into new second best and third best programs. At SWRL we 
deliberately, directly, and consciously label these distinct generations 

of effort. 

T is the now generation. It is always something we have become 
tired - of; but it is nearly ready to let go of, and it does represent 
specifiable advances over past state-of-the-art alternatives. 

T+l is the model that commands the greatest proportionate staff 
effort at any given time since its development is concurrently active 
at many stages. It is on the way to becoming the new model T. 

T+2 is the wishful model. It represents the cumulation of the 
activities that staff think they would really like to be doing, 
rather than working on the Tfl or T model. Some of this wishfulness 
adventitiously vanishes over time. Some is washed out in subsequent 
empirical^f forts. And some turn out to provide the specifications 
for the forthcoming T+l model. 
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Introducing these time considerations does at least two things. 
First, it gets products out, meaning it provides the conditions nec- 
essary to get people to let go of the results of their efforts not 
an easy task. Second, it contributes to staff mental healthy One of 
our biggest personal adjustment problems is what I call the whipsaw 
dilemma." Inevitably, before a given development effort is completed, 
its limitations and defects are clear to all who have contributed to 
it. Each staff member tends to assign himself personal blame for thes 
limitations, not realizing first that the ability to identify the 
limitations is probably limited to those intimately involved in the 
development, that the anticipated dire consequences of the limitations 
may well be overestimated, and finally, that the removal of these 
limitations is the basis for one's job in the future. This self- 
defeating mechanism is undone by stretching out from first best to 
third best. 

Three other facets of Table 2 warrant attention. The first is 
the emphasis on specifications. The essence of development is iter- 
ative feedback. But this has usually been perceived in education as 
a test-revise-retest cycle. 
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This is a non— analytic , uncontrolled, and inefficient paradigm. Tre- 
mendous gains are made by adding a specifications box to the paradigm 
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lt is erroneous to infer that specifications are generated cleanly 
and simply or that they are originally conceived in the summary form xn 
which they may be later stated. Most of the work in preparing specifi- 
cations is thrown away or is synthesized from much loader paper work^ 
The point is that it is more reasonable to key on spec 
basis for revision rather than on the product per se. Documentation 
thus becomes a, if not the, prime concern in programmatic instructxo 
development . 

A second point revolves around the concept of subset optimization. 

It is both reasonable and necessary to use subset criter a. or e ^ m _ 
pie, although specified changes in pupil behavior represent the crite 
rion which instructional development is attempting to optimize, it is 
very unwise to use this as the sole feedback basis. For example, our 
staff have, at times, been very disappointed when the introduction of 
specific procedures such as teacher training, audxovisual segments , etc. , 
have not improved pupil performance. One could throw out the prototype 
and look in a different area. But on closer examination and analysts, 
we have each time determined that the intended function was itself not 
being performed by the instructional component. That is, the teachers 
learned nothing or the wrong things from the training, the audiovisual 
segments were being "misused," etc. With subcomponents performing 
optimally, one has a much better likelihood of accomplishing larger 
functions optimally. Optimization is unlikely achievable with unreli 
able subcomponents. This sounds obvious, but xt is almost uni\ ers^l y 
overlooked in education. 

The final point involves the distinction between comparative 

and cumulative experimentation. The experimental tradxtxonxn the 

behavior sciences is comparative. One compares effects of dxfferent 
phenomena introduced concurrently or simultaneously. An equally 
venerable experimental tradition involves comparisons over time whxch 
cumulate in more optimal performance. This tradition has °f ten ; 
rejected in education because of industrial connotations. Cumulatxve 
optimization methodology can, however, be applied to educational 
endeavors without considering people as machines; just as comparative 
experiment methodology may be applied without consxdering peop e as 
fertilizer. 

Two things have impressed me in our use of the cumulative optimiza 
tion approach at SWRL. The first is the high degree of replicability 
that is possible. If one is concerned with reducing "treatment variance 
rather than analyzing it, the signal-noise ration very qu c y can e 
improved. This has been very evident in the criterion-referenced 
measurement of pupil performance. The differential difficulty o a 
set of skills is very reliable. That is, word attack skills^are more 
difficult than word recognition. Word selection skills are more 
difficult than construction skills, and so on. This differential 
difficulty holds up very well across extreme differences in personal 
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characteristics of pupils and in ecological characteristics of schools. 
This is highly useful information in a development context , since j_t 
permits the differential allocation of uncertainty reducing efforts 
to shore up dif ficult- to-attain areas while maintaining performance 
in "no problem" areas. 

The second impressive thing has been the actual attainment of 
reliable increments in the instructional effectiveness of a given 
program against specified objectives. This incremental improvement 
has involved full-year programs under hands-off tryout conditions. 

X had had faith that such improvement could be demonstrated, but X 
must admit that after so much previous experience with NSD compare 
tive experimentation, I was holding my breath the second year. 

Programmatic instructional development by definition requires 
continuous directed allocation of personnel and financial resources. 

SWRL planning is within a three-five year time frame. That is , we 
arbitrarily limit the time to reduce an effort from T+l to T_ and out 
to no more than five years. This is pressing it and is due to the 
still unreasonable pressure from both the schools and the Federal 
Government to "produce quickly." A four-seven year time frame would 
be more reasonable and less costly in the long run. This does not 
mean that it takes four-seven years to "peek" at the results. e 
velopment is inherently inefficient compared with production; the 
bulk of the total work output of a development effort is by intent 
discarded. But the small, steady steps of uncertainty reduction 
which are cumulated are clear from the beginning. If they are not , 
the development effort is a boondoggle* 

By definition also, any development effort involves a continuous 
risk of failure. This risk is also characteristic of the overall enter 
prise of instructional development at the present time. Programmatic 
instructional development has not yet fully paid off demonstrably, 
but the risk is being rapidly and successively reduced. c eva e 
returns now appear to be extremely high and the risk low for both 
personal and societal levels of involvement* 
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